IPIP-0476: Delegated Routing DHT Closest Peers API #476

achingbrain · 2024-06-06T07:42:34Z

Adds a new HTTP endpoint that can be used to request records for the closest peers to a given key that the routing implementation knows about.

The use-case for this is browser nodes performing random walks to find peers that they can make a circuit relay reservation on, without having to be DHT clients to perform the walk which can be undesirable given all the connection/processing overhead that entails.

I've tried to avoid defining what "closest" means to leave it up to the routing implementation, only that the responses should be "closer" than the optional closerThan parameter which should mean the caller gets useful results for other scenarios.

Adds a new HTTP endpoint that can be used to request records for the closest peers to a given key that the routing implementation knows about. The use-case for this is browser nodes performing random walks to find peers that they can make a circuit relay reservation on, without having to be DHT clients to perform the walk which can be undesirable given all the connection/processing overhead that entails.

src/routing/http-routing-v1.md

Co-authored-by: Guillaume Michel <[email protected]>

SgtPooki · 2025-04-17T14:33:41Z

Just connecting some dots. This problem this solves seems somewhat related to libp2p/specs#222

achingbrain · 2025-05-28T15:27:12Z

Refs ipshipyard/roadmaps#16

update endpoint path to /routing/v1/dht/closest/{peer-id} as agreed in PR review comments

- create new "DHT Routing API" section for DHT-specific operations - move /routing/v1/dht/closest/{peer-id} to the new DHT section - keep general peer lookup in "Peer Routing API" section - update cache value to 172800 (48h) for consistency - fix typo: "Query Paramters" -> "Query Parameters"

rename /routing/v1/dht/closest/{peer-id} to /routing/v1/dht/closest/peers/{peer-id} for future-proofing, as we may add API for querying entire keyspace in the future also update document date to 2025-08-19

documents the new /routing/v1/dht/closest/peers/{peer-id} endpoint that enables lightweight peer discovery for browser nodes and other resource-constrained clients without requiring full DHT participation

keep 2025-08-19 date from PR branch

github-actions · 2025-08-19T18:25:53Z

🚀 Build Preview on IPFS ready

🔎 Commit: d551c94
🔏 CID bafybeiaybdf53eohqwjecfa6e6hb3adqbyidq2quom3bwkgcizl5blh4aa
📦 Preview:

reorganize sections for better logical flow: - Content Routing API - Peer Routing API - IPNS API - DHT Routing API (moved here)

lidel · 2025-08-19T18:30:54Z

@achingbrain FYSA to make this bit more formal, generated IPIP-0476 documenting why we need this: it includes the motivation from the PR (browser nodes needing lightweight peer discovery), the specification, rationale, and future-proofing considerations discussed in the PR comments.

Feel free to add any context to the document you find useful.

This adds the SERVER-side for GetClosestPeers. Since FindPeers also returns PeerRecords, it is essentially a copy-paste, minus things like addrFilters which don't apply here, plus `count` and `closerThan` parsing from the query URL. The tests as well. We leave all logic regarding count/closerThan to the ContentRouter (the DHT, or the Kubo wrapper around it). Spec: ipfs/specs#476

src/ipips/ipip-0476.md

hsanjuan · 2025-09-03T07:53:56Z

src/ipips/ipip-0476.md

+  - If omitted, the routing implementation should use its own peer ID
+- `count` (optional): Number of peer records to return
+  - Minimum 1, maximum 100, default 20
+


Unlike the FindPeers endpoint, the GetClosestPeers endpoint does not support protocol or address filters.

I understand this is a DHT-specific method and everything-DHT is unknown, but if that might change in the future, we may introduce support for the filters now from the beginning.

and everything-DHT is unknown

Not strictly true since the endpoint may know more information. For example https://github.com/ipfs/someguy learns more about peers on the network which helps it limit sending unhelpful peers to web browsers.

The implementation of ipfs/specs#476 suggests that content routers should support a DHT-specific operations (GetClosestPeers). Content routers depend on routing interfaces defined in the `routing` package and some decorator interfaces defined here. So it seems like the natural place to add yet another interface for this type of router.

guillaumemichel · 2025-09-04T12:14:33Z

src/ipips/ipip-0476.md

+
+Currently, to find peers close to a particular key in the DHT keyspace, a node must:
+1. Be a full DHT client with all the associated overhead
+2. Maintain connections to many peers


In theory this isn't necessary.

It is true that many connections are necessary to perform a lookup, but a client technically doesn't need to maintain any connection when not performing a lookup actively. It should remember at least some addresses of DHT servers (e.g bootstrappers are enough).

I think we just miss light DHT client implementations.

src/ipips/ipip-0476.md

This allows Kubo to respond to the GetClosestPeers() http routing v1 endpoint as spec'ed here: ipfs/specs#476 It is based on work from ipfs/boxo#1021 We let IpfsNode implmement the contentRouter.Client interface with the new method. We use our DHTs to get the closest peers. We try to respect the count/closerThan options here. We then trigger FindPeers lookups to fill-in information about the peers (addresses) and return the result. Tests are missing and will come up once discussions around the spec and the boxo pr have settled.

ref: #476 (comment) Co-authored-by: Hector Sanjuan <[email protected]>

This adds the SERVER-side for GetClosestPeers. Since FindPeers also returns PeerRecords, it is essentially a copy-paste, minus things like addrFilters which don't apply here, plus `count` and `closerThan` parsing from the query URL. The tests as well. We leave all logic regarding count/closerThan to the ContentRouter (the DHT, or the Kubo wrapper around it). Spec: ipfs/specs#476

This allows Kubo to respond to the GetClosestPeers() http routing v1 endpoint as spec'ed here: ipfs/specs#476 It is based on work from ipfs/boxo#1021 We let IpfsNode implmement the contentRouter.Client interface with the new method. We use our DHTs to get the closest peers. We try to respect the count/closerThan options here. We then trigger FindPeers lookups to fill-in information about the peers (addresses) and return the result. Tests are missing and will come up once discussions around the spec and the boxo pr have settled.

guillaumemichel

It would be useful to extend the scope of the API to allow getting the closest DHT servers to CIDs and IPNS keys in addition to Peer IDs.

src/routing/http-routing-v1.md

This allows Kubo to respond to the GetClosestPeers() http routing v1 endpoint as spec'ed here: ipfs/specs#476 It is based on work from ipfs/boxo#1021 We let IpfsNode implmement the contentRouter.Client interface with the new method. We use our DHTs to get the closest peers. We try to respect the count/closerThan options here. We then trigger FindPeers lookups to fill-in information about the peers (addresses) and return the result. Tests are missing and will come up once discussions around the spec and the boxo pr have settled.

changed path parameter from {peer-id} to {key} to accept both CIDs and Peer IDs, matching actual DHT usage where closest peers can be queried for arbitrary keys removed count and closer-than query parameters that were adding complexity without clear use cases in practice clarified response size should match DHT bucket size (20 for Amino DHT) added note that this optional endpoint helps light clients lower the cost of DHT walks in browser contexts

…lt (#10954) This allows Kubo to respond to the GetClosestPeers() http routing v1 endpoint as spec'ed here: ipfs/specs#476 It is based on work from ipfs/boxo#1021 We let IpfsNode implmement the contentRouter.Client interface with the new method. We use our WAN-DHT to get the closest peers. Additionally, Routing V1 HTTP API is exposed by default which enables light clients in browsers to use Kubo Gateway as delegated routing backend Co-authored-by: Marcin Rataj <[email protected]>

Final ratification cleanup aligning specs with actual shipped implementation. Changes to IPIP-0476: - change path parameter from {peer-id} to {key} - remove unimplemented query parameters (count, closer-than) - document query parameters in Alternatives section with rationale - add peer sorting requirement (XOR distance for Kademlia DHTs) - update test fixtures to reflect actual API surface - note about raw codec for arbitrary multihash lookups - update date to 2025-11-20 Changes to http-routing-v1.md: - add raw codec documentation for arbitrary multihash lookups - clarify peer sorting requirement (SHOULD, XOR distance) - fix streaming description (remove "more results" language) - add note about streaming being blocked until DHT lookup completes - reorganize frontmatter: move inactive editors to former_editors - add contributors to thanks section - update date to 2025-11-20 Both specs now accurately reflect the API shipped in: - boxo v0.35.2 (ipfs/boxo#1021) - someguy v0.11.0 (https://github.com/ipfs/someguy/releases/tag/v0.11.0)

lidel

Ratified and shipped in someguy v0.11.0

Thank you to everyone else who contributed to this IPIP through design discussions and implementation work.

The reference implementation of IPIP-476 has shipped in someguy v0.11.0 (released 2025-11-20) and is now deployed to the public delegated routing endpoint at https://delegated-ipfs.dev/routing/v1 (documented at
https://docs.ipfs.tech/concepts/public-utilities/#delegated-routing-endpoint).

What shipped:

Endpoint: GET /routing/v1/dht/closest/peers/{key}
Accepts both CID and Peer ID formats
Returns up to 20 DHT-closest peers (Amino DHT bucket size)
Peers sorted by XOR distance
No query parameters in MVP (simplified for initial deployment)

Implementation references:

boxo v0.35.2: ipfs/boxo#1021
someguy v0.11.0: https://github.com/ipfs/someguy/releases/tag/v0.11.0

The specs have been updated to reflect what actually shipped.
I've addressed the unresolved discussion threads above with resolution notes.

Merging as ratified – further changes can be made by opening a new IPIP.

all pending discussions got resolved

achingbrain force-pushed the feat/add-get-closest-peers-to-http-routing branch from a6bf57c to 7e4489e Compare June 6, 2024 07:42

achingbrain force-pushed the feat/add-get-closest-peers-to-http-routing branch from 7e4489e to 0a3d8e3 Compare June 6, 2024 07:43

achingbrain requested a review from lidel June 6, 2024 07:48

lidel reviewed Jun 8, 2024

View reviewed changes

src/routing/http-routing-v1.md Outdated Show resolved Hide resolved

guillaumemichel reviewed Jun 11, 2024

View reviewed changes

src/routing/http-routing-v1.md Outdated Show resolved Hide resolved

src/routing/http-routing-v1.md Outdated Show resolved Hide resolved

src/routing/http-routing-v1.md Outdated Show resolved Hide resolved

src/routing/http-routing-v1.md Outdated Show resolved Hide resolved

aschmahmann previously requested changes Jun 17, 2024

View reviewed changes

src/routing/http-routing-v1.md Outdated Show resolved Hide resolved

src/routing/http-routing-v1.md Outdated Show resolved Hide resolved

docs: apply suggestions from code review

351b8a7

Co-authored-by: Guillaume Michel <[email protected]>

aschmahmann mentioned this pull request Oct 23, 2024

TODOs before someguy's General Availability ipfs/someguy#24

Open

11 tasks

achingbrain mentioned this pull request Apr 30, 2025

feat!: add ipns reproviding/republishing ipfs/helia#764

Merged

4 tasks

hsanjuan assigned hsanjuan and unassigned hsanjuan Aug 18, 2025

hsanjuan mentioned this pull request Aug 18, 2025

http-routing: add "get closest peers" operation ipfs/boxo#1004

Closed

fix: use agreed path for closest peers endpoint

9aa55a3

update endpoint path to /routing/v1/dht/closest/{peer-id} as agreed in PR review comments

lidel changed the title ~~http-routing: add "get closest peers" operation~~ IPIP-476: http routing: add "get closest peers" kad-dht operation Aug 19, 2025

lidel added 3 commits August 19, 2025 19:42

feat: future-proof DHT API with /peers/ segment

ab45d97

rename /routing/v1/dht/closest/{peer-id} to /routing/v1/dht/closest/peers/{peer-id} for future-proofing, as we may add API for querying entire keyspace in the future also update document date to 2025-08-19

feat: add IPIP-0476 for DHT closest peers API

7c79e02

documents the new /routing/v1/dht/closest/peers/{peer-id} endpoint that enables lightweight peer discovery for browser nodes and other resource-constrained clients without requiring full DHT participation

lidel requested a review from a team as a code owner August 19, 2025 18:18

merge: resolve conflict with main branch

b278b84

keep 2025-08-19 date from PR branch

refactor: move DHT Routing API after IPNS API

4bdfa84

reorganize sections for better logical flow: - Content Routing API - Peer Routing API - IPNS API - DHT Routing API (moved here)

lidel changed the title ~~IPIP-476: http routing: add "get closest peers" kad-dht operation~~ IPIP-0476: Delegated Routing DHT Closest Peers API Aug 19, 2025

hsanjuan approved these changes Sep 3, 2025

View reviewed changes

hsanjuan mentioned this pull request Sep 3, 2025

routing/http: add support for GetClosestPeers (IPIP-476) ipfs/boxo#1021

Merged

hsanjuan mentioned this pull request Sep 3, 2025

Add DHTRouter interface libp2p/go-libp2p-routing-helpers#93

Open

guillaumemichel reviewed Sep 4, 2025

View reviewed changes

hsanjuan mentioned this pull request Sep 5, 2025

feat: support GetClosesPeers (IPIP-476) and ExposeRoutingAPI by default ipfs/kubo#10954

Merged

3 tasks

fix: use kebab-case for query parameter naming

dd86eb3

ref: #476 (comment) Co-authored-by: Hector Sanjuan <[email protected]>

guillaumemichel reviewed Sep 26, 2025

View reviewed changes

src/routing/http-routing-v1.md Outdated Show resolved Hide resolved

lidel mentioned this pull request Oct 17, 2025

feat: add "getclosestpeers" support (IPIP-476) ipfs/someguy#124

Merged

1 task

lidel added 2 commits November 20, 2025 19:36

Merge branch 'main' into feat/add-get-closest-peers-to-http-routing

cc70f9b

lidel requested a review from aschmahmann November 20, 2025 19:31

lidel added 2 commits November 20, 2025 20:46

chore(ipip-0476): mark as ratified

3bfb193

docs(routing): clarify DHT API is optional, accept 404 or 501

d551c94

lidel approved these changes Nov 20, 2025

View reviewed changes

lidel merged commit a5b6db6 into main Nov 20, 2025
5 checks passed

lidel deleted the feat/add-get-closest-peers-to-http-routing branch November 20, 2025 20:04

IPIP-0476: Delegated Routing DHT Closest Peers API #476

IPIP-0476: Delegated Routing DHT Closest Peers API #476

Uh oh!

Conversation

achingbrain commented Jun 6, 2024

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

SgtPooki commented Apr 17, 2025

Uh oh!

achingbrain commented May 28, 2025

Uh oh!

github-actions bot commented Aug 19, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

🚀 Build Preview on IPFS ready

Uh oh!

lidel commented Aug 19, 2025

Uh oh!

Uh oh!

hsanjuan Sep 3, 2025

Choose a reason for hiding this comment

Uh oh!

aschmahmann Sep 3, 2025

Choose a reason for hiding this comment

Uh oh!

guillaumemichel Sep 4, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

guillaumemichel left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

lidel left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Ratified and shipped in someguy v0.11.0

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

7 participants

github-actions bot commented Aug 19, 2025 •

edited

Loading

lidel left a comment •

edited

Loading